System and method for automatic camera hand-off using location measurements

ABSTRACT

Systems and methods for receiving video images from a plurality of video cameras having respective fields of view that cover a geographical region. At a first time, first video images of an object may be output, where the images are captured by a first video camera selected from the plurality of video cameras. Location indications may be received, which specify a geographical location of the object in the geographical region and which are determined independently of the video images. At a second time subsequent to the first time, a second video camera from the plurality may be selected based on the location indications. The output may be switched to the second video images of the object, which are captured by the selected second video camera.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 12/887,089, filed on Sep. 21, 2010 and entitled “System andmethod for automatic camera hand-off using location measurements,” thedisclosure of which is incorporated herein by reference in its entirety.

FIELD OF THE DISCLOSURE

The present disclosure relates generally to video surveillance, andparticularly to methods and systems for performing hand-off betweenvideo cameras.

BACKGROUND OF THE DISCLOSURE

Video surveillance systems are deployed and operated in variousapplications, such as airport security, crime prevention and accesscontrol. In a typical video surveillance application, multiple videocameras acquire video footage, which is viewed and/or recorded at amonitoring center.

Mobile communication networks deploy various techniques for measuringthe geographical locations of wireless communication terminals. Suchtechniques are used, for example, for providing Location Based Services(LBS) and emergency services in cellular networks. Some locationtracking techniques are based on passive probing of network eventsgenerated by the wireless terminals. Other techniques are active, i.e.,proactively request the network or the terminal to provide locationinformation.

SUMMARY OF THE DISCLOSURE

An embodiment that is described herein provides a method, including:

-   -   receiving video images from a plurality of video cameras having        respective fields of view that cover a geographical region;    -   at a first time, outputting first video images of an object,        which are captured by a first video camera selected from the        plurality;    -   receiving location indications, which specify a geographical        location of the object in the geographical region and which are        determined independently of the video images;    -   at a second time subsequent to the first time, selecting based        on the location indications a second video camera from the        plurality, different from the first video camera; and    -   switching to output second video images of the object, which are        captured by the selected second video camera.

In some embodiments, the object includes an individual moving throughthe geographical region. Additionally or alternatively, the object isassociated with a wireless communication terminal that communicates witha communication network, and receiving the location indications includesreceiving location measurements of the wireless communication terminalfrom the communication network. In some embodiments, selection of thesecond video camera is performed by a switch in the communicationnetwork. In a disclosed embodiment, outputting the first and secondvideo images includes displaying the first and second video images to anoperator.

In an embodiment, selecting the second video camera includes querying apredefined mapping of geographical locations to image locations in thefields of views of the video cameras. In another embodiment, selectingthe second video camera includes calculating, based on the locationindications, respective first and second image locations of the objectin first and second fields of view of the first and second videocameras, and selecting the second camera based on the first and secondimage locations. In yet another embodiment, selecting the second videocamera further includes controlling the second video camera so as tomodify a respective field of view of the second video cameraresponsively to the location indications.

There is additionally provided, in accordance with an embodiment that isdescribed herein, a system, including:

-   -   a first interface, which is configured to receive video images        from a plurality of video cameras having respective fields of        view that cover a geographical region;    -   a second interface, which is configured to receive location        indications, which specify a geographical location of an object        in the geographical region and which are determined        independently of the video images; and    -   a processor, which is configured to output, at a first time,        first video images of the object, which are captured by a first        video camera selected from the plurality, to select from the        plurality based on the location indications, at a second time        subsequent to the first time, a second video camera different        from the first video camera, and to switch to output second        video images of the object, which are captured by the selected        second video camera.

The present disclosure will be more fully understood from the followingdetailed description of the embodiments thereof, taken together with thedrawings in which:

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a pictorial, schematic illustration of a video surveillancesystem, in accordance with an embodiment of the present disclosure;

FIG. 2 is a block diagram that schematically illustrates a videosurveillance system, in accordance with an embodiment of the presentdisclosure;

FIG. 3 is a diagram that schematically illustrates an operator displayin a video surveillance system, in accordance with an embodiment of thepresent disclosure; and

FIG. 4 is a flow chart that schematically illustrates a surveillancemethod, in accordance with an embodiment of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS Overview

Video surveillance systems typically collect video images from multiplevideo cameras and present the images to an operator. The operator mayobserve objects (e.g., individuals) of interest in the images and takeappropriate actions. In a typical video surveillance system, each camerahas a limited field-of-view, and the multiple cameras collectively covera certain region of interest.

In many practical scenarios, an individual of interest is in motion, andin particular moves in and out of the fields-of-view of differentcameras. The operator, on the other hand, is concerned with tracking theindividual in question as seamlessly as possible. Tracking an individualin motion therefore requires hand-off between cameras, i.e., modifyingthe selection of the camera whose video is presented to the operator inan attempt to track the individual of interest without interruption.

Embodiments that are described hereinbelow provide improved methods andsystems for automatic hand-off between video cameras. In someembodiments, a correlation processor receives location indications,which indicate the geographical location of the individual of interest.The correlation processor decides when to perform camera hand-off, andto which camera, based on the estimated location of the individual asreflected by the location indications.

The location indications used for triggering camera hand-off aredetermined independently of the video images, i.e., do not rely on imageprocessing in any way. For example, the individual may carry a cellularphone or other wireless terminal, which communicates with a cellularnetwork that also measures the terminal's location. Locationmeasurements of the terminal, provided by the cellular network, mayserve as location indications of the terminal's user for the purpose ofcamera hand-off.

Since the disclosed techniques use location indications that areindependent of the video images, they are able to perform camerahand-off reliably under difficult visual conditions, e.g., when thecamera's field-of-view is obstructed, when video quality is poor or whenthe fields-of-view of neighboring cameras do not overlap.

The embodiments described herein mainly address video surveillance ofindividuals that carry wireless communication terminals, and performingcamera hand-off based on location measurements of the terminals.Nevertheless, the methods and systems described herein can also be usedwith various other types of tracked objects and location indications,such as in tracking objects that are fitted with Radio FrequencyIdentification (RFID) tags, or Automatic Vehicle Location (AVL) orAutomatic Person Location (APL) transponders.

System Description

FIG. 1 is a pictorial, schematic illustration of a video surveillancesystem 20, in accordance with an embodiment that is described herein.System 20 tracks individuals of interest within a certain geographicalregion 22, which may comprise a city, city center, neighborhood, airportterminal or any other suitable area. Systems of this sort may beoperated, for example, by law enforcement or other Government agencies,such as in airport security systems, crime prevention systems oranti-terrorism systems.

System 20 tracks individuals 24 using a video subsystem 36, whichcomprises multiple video cameras. In the present example, subsystem 36comprises five video cameras 32A . . . 32E, although any other number ofcameras may be used. Each camera has a certain field-of-view, whichcovers a particular sector in area 22. The cameras capture video imagesof their respective sectors and send the images to subsystem 36. Thecameras may have fixed fields-of-view, or they may comprise cameraswhose fields-of-view are adjustable such as Pan-Tilt-Zoom (PTZ) cameras.

At least some of individuals 24 communicate with a mobile communicationnetwork 40 by operating wireless communication terminals 28.(Individuals 24 are therefore sometimes referred to herein as users. Thetwo terms are used interchangeably.) Terminals 28 may comprise, forexample, cellular phones, wireless-enabled computers or Personal DigitalAssistants (PDAs), or any other suitable communication or computingdevice having wireless communication capabilities. Communication network40 and terminals 28 may conform to any suitable communication standardor protocol, such as Long Term Evolution (LTE), Universal MobileTelecommunication System (UMTS), CDMA2000 or other third generation (3G)cellular standard, Global System for Mobile communication (GSM) orIntegrated Digital Enhanced Network (IDEN). Alternatively, the networkand terminals may conform to the IEEE 802.16 (WiMAX) standards or otherwireless data standard. Although FIG. 1 shows only a single user for thesake of clarity, practical networks typically communicate with a largenumber of users and terminals. Although the description that followsrefers to a single network, system 20 may operate with any desirednumber of communication networks, which may conform to differentstandards or protocols.

System 20 comprises a location tracking subsystem 44, which measures thegeographical locations of wireless communication terminals 28 in area22. The example of FIG. 1 refers to a single location trackingsubsystem. Alternatively, the system may comprise two or more locationtracking subsystems, which may be of different types. Location trackingsubsystem 44 may apply any suitable location tracking techniqueavailable in the network, or a combination of such techniques, in orderto measure terminal locations.

Some location tracking techniques, referred to as network-basedtechniques, are carried out by base stations and other network-sidecomponents of the network, without necessarily using special hardware orsoftware in terminals 28. Other location tracking techniques areterminal-based, i.e., use special hardware or software in wirelessterminals 28. Terminal-based techniques may comprise, for example,techniques that use Global Navigation Satellite Systems (GNSS) such asGPS or GALILEO. The location tracking techniques may be passive oractive. Passive techniques perform unobtrusive probing of the signalinginformation transmitted in the network, and extract location informationfrom the monitored signaling. Active techniques, on the other hand,proactively request the network or the terminal to provide locationinformation.

Some examples of location tracking techniques that can be used for thispurpose are described in U.S. patent application Ser. No. 12/497,799,filed Jul. 6, 2009, which is assigned to the assignee of the presentpatent application and whose disclosure is incorporated herein byreference. Location tracking subsystem 44 thus measures the geographicallocations of at least some of terminals 28, and produces locationindications that indicate the measured terminal locations.

System 20 comprises a correlation system 48, which interacts withlocation tracking subsystem 44 and video subsystem 36. For a givenindividual 24, correlation system 48 selects the appropriate videocamera for tracking this individual based on the location indications ofterminal 28 operated by this individual, as provided by subsystem 44. Inparticular, the correlation system uses the location indications todecide when to perform camera hand-off (i.e., switch to a differentcamera), and to which camera. In a typical implementation, correlationsystem 48 notifies video subsystem 36 of the estimated geographicallocation of the individual, and the video subsystem selects the camerathat is best suited for viewing this location. Alternatively, the cameraselection may be carried out by the correlation system itself.

The video from the selected video camera is provided from videosubsystem 36 via correlation system 48 to a monitoring center 52. Thevideo is typically displayed to an operator 56 using an output devicesuch as a display 60. In the example of FIG. 1, operator 56 is presentedwith real-time video images showing user 24 and his or her vicinity.Operator 56 may control the display or provide other input using aninput device 64, such as a keyboard or mouse. Additionally oralternatively to presenting the video to operator 56, correlation system48 or monitoring center 52 may store the video for later retrieval andanalysis.

FIG. 2 is a block diagram that schematically illustrates components ofsystem 20, in accordance with an embodiment that is described herein.Video subsystem 36 comprises a networked video server 70, which managesthe operation of cameras 32, receives the video images captured by thecameras and sends the video to correlation system 48. Video server 70stores captured video in a video records database 74 for off-lineviewing and analysis. Subsystem 36 also comprises an image-to-locationmapping database 78. Database 78 stores a predefined mapping of imagecoordinates to geographical coordinates for each camera 36. By queryingthis database with a certain geographical location, server 70 candetermine which of cameras 32 has a field-of-view that covers thisgeographical location.

Correlation system 48 comprises interfaces 82 and 86 for communicatingwith location tracking subsystem 44 and video subsystem 36,respectively. System 48 further comprises a correlation processor 90,which carries out the correlation functions described herein.

Monitoring center 52 comprises a Location-Based Monitoring (LBM) server94, which accepts the video from the selected camera (or cameras) fromcorrelation system 48 and presents it to operator 56 using an operatorterminal 98. In the example of FIG. 2, LBM server 94 and correlationsystem 48 interact directly. In alternative embodiments, however, system48 and server 94 may interact via a database that stores the selectedvideo (e.g., database 74 or a separate database). In some embodiments,server 94 interacts with a Geographic Information System (GIS) 102,which provides map information and other geographic data forpresentation purposes. The GIS may hold any suitable kind of geographicinformation, such as Points of Interest (POIs), clutter data andblueprints of area 22. The geographic information is stored in a mapdatabase 106.

The configurations of FIGS. 1 and 2 are example configurations, whichwere selected purely for the sake of conceptual clarity. In alternativeembodiments, any other suitable system configuration can also be used.For example, correlation system 48 and monitoring center 56 may becollocated, and the functions of processor 90 can be integrated intoserver 94. In some embodiments, some functions of correlation system 52may be implemented as part of a switch, such as a Mobile SwitchingCenter (MSC), of communication network 40.

The different databases in system 20 (e.g., databases 74, 78 and 106)may be implemented using any suitable data structures and storagedevices. Typically, processor 90 and server 94 comprise general-purposecomputers, which are programmed in software to carry out the functionsdescribed herein. The software may be downloaded to the computers inelectronic form, over a network, for example, or it may, alternativelyor additionally, be provided and/or stored on tangible media, such asmagnetic, optical, or electronic memory.

Automatic Camera Hand-Off Process

FIG. 1 above shows an example scenario, in which user 24 moves througharea 22 between three locations denoted X0, X1 and X2. When the user isat location X0 he is best viewed by camera 32A, when the user is atlocation X1 he is best viewed by camera 32D, and when the user is atlocation X2 he is best viewed by camera 32E. In this example, videosurveillance of this user should begin with camera 32A, then hand-off tocamera 32D, and finally hand-off to camera 32E. Note that some locationsalong the user's path are obstructed from view by buildings and cannotbe viewed by any of the cameras.

As user 24 moves through area 22, location tracking subsystem 44measures the geographical location of terminal 28 and sends locationindications to correlation system 48. Note that the user need notnecessarily operate the terminal, e.g., conduct calls. It is usuallysufficient that the terminal is switched on. Correlation system 48 sendsthe estimated geographical location of the user to video subsystem 36.Based on the user's estimated location, and on the known fields-of-viewof the different cameras, server 70 decides which camera is best suitedfor viewing the user.

Typically, server 70 queries database 78 with the user's estimatedgeographical location, so as to determine one or more cameras whosefield-of-view currently covers the user. If the user appears in thefields-of-view of multiple cameras, server 70 may apply any suitablecriteria or logic to select one of the cameras. For example, server 70may select the camera in which the user appears closest to the center ofthe field-of-view, or the camera that is geographically closest to theuser. The video from the selected camera is sent to monitoring center 52and presented to operator 56.

The above camera selection process is typically carried out continually,in real-time. In other words, as user 24 moves through area 22, locationsubsystem 44 sends up-to-date location indications to correlation system48, and the correlation system sends up-to-date location estimates tovideo subsystem 36. The video subsystem updates the camera selection(i.e., performs camera hand-off) whenever needed to match the changinglocation estimates, and sends the video captured by thecurrently-selected camera.

FIG. 3 is a diagram that schematically illustrates the video displayedto operator 56 in the above example scenario, in accordance with anembodiment that is described herein. The left hand side of the figureshows the cameras selected by server 70 as user 24 moves from locationx0 to location X1 and then to location X2. When the user is in thevicinity of x0 around a time denoted t0, server 70 selects camera 32A(presented as “CAMERA A” to the operator). When the user moves away fromx0 and approaches X1 around a time denoted t1, server 70 hands-off tocamera 32D (“CAMERA B”). When the user approaches X2 around a timedenoted t2, server 70 hands-off to camera 32E (“CAMERA C”).

The video displayed to operator 56 in this scenario is shown on theright hand side of FIG. 3. Displays 110A . . . 110C show the video thatis presented to the operator around times t0 . . . t2, respectively.Around time t0, user 24 is located in the vicinity of location x0, andtherefore system 20 displays the video captured by camera A (32A) at thecenter of the screen. This view is shown in display 110A. In the presentexample, the system also displays a smaller window, showing the video ofcamera B (32D) to which hand-off is anticipated.

Between t0 and t1, the user moves away from x0 and approaches X1. Atsome point in time between t0 and t1, server 70 decides to hand-off fromcamera A (32A) to camera B (32D). As a result, around time t1 the systemdisplays the video of camera B (32D) at the center of the screen, asshown in display 110B. In this example, the display also shows smallerwindows with the video of camera A (32A) from which the previoushand-off was performed, and of camera C (32E) to which the next hand-offis anticipated.

Between t1 and t2, the user moves away from X1 toward X2. At a certaintime between t1 and t2, server 70 decides to hand-off from camera B(32D) to camera C (32E). Therefore, around time t2 system 20 displaysthe video of camera C (32E) at the center of the screen, as shown indisplay 110C. Display 110C also shows a smaller window with the video ofcamera B (32D) from which the previous hand-off was performed.

As can be appreciated, the above camera hand-off process is performed bysystem 20 automatically without operator intervention. At any point intime, system 20 selects the video camera that best covers the currentgeographical location of the user, based on the location measurementsperformed on the user's communication terminal. As a result, theoperator is automatically presented with continuous video footage of theuser's location, even though the user moves in and out of thefields-of-view of different video cameras.

The user interface shown in FIG. 3 is an example user interface, whichis shown purely for the sake of conceptual clarity. In alternativeembodiments, any other suitable user interface can also be used topresent the video from the selected camera or cameras to the operator.

Surveillance Method Description

FIG. 4 is a flow chart that schematically illustrates a surveillancemethod, in accordance with an embodiment that is described herein. Themethod describes the process of tracking a given user of interest,referred to as a target user, as he or she moves through area 22. Themethod begins with LBM server 94 displaying to operator 56 video from acertain video camera, at a video output step 120. The camera has beenselected by server 70 based on location indications of the target user'scommunication terminal (e.g., cellular phone) provided by locationtracking subsystem 44.

Correlation system 48 receives up-to-date location indications fromsubsystem 44, at a location updating step 124. In response to thelocation indications, system 48 sends up-to-date estimates of the targetuser's geographical location to video server 70.

Based on the estimated user location, server 70 evaluates whether it isnecessary to perform camera hand-off (i.e., switch to displaying thevideo of a different camera), at a switching evaluation step 128. Server70 may apply various conditions or criteria for deciding whether toperform camera hand-off, and to which camera. In some embodiments, thecriterion is defined with respect to the user's image location withinthe field-of-view of the currently-selected camera, and/or the user'simage location within the fields-of-view of neighboring cameras. Note,however, that server 70 determines these image locations using thelocation indications and using database 78, i.e., irrespective of thevideo images themselves. Server 70 typically does not rely on imageprocessing to determine the user's image position in the cameras'fields-of-view.

For example, server 70 may check whether the user's estimated locationcorresponds to the edge of the currently-selected camera'sfield-of-view. If the user's location is in the middle of the currentcamera's field-of-view, hand-off may not be necessary. Otherwise, server70 may attempt to find a camera that better covers the user's location.In some embodiments, the decision to switch to a different camera isbased only on the estimated geographical location of the user,regardless of the user's position at the camera's field-of-view.Additionally or alternatively, server 70 may apply any other suitablecriterion to decide whether, and to which camera, to perform hand-off.

Server 70 checks whether hand-off is needed, at a checking step 132. Ifno hand-off is needed, the method loops back to step 120 above, andsystem 20 continues to display video from the currently-selected camera.If hand-off is needed, server 70 performs camera hand-off, at a hand-offstep 136. Server 70 selects a camera whose field-of-view better coversthe current user location, as explained above. The method loops back tostep 120 above, and system 20 begins to display video from thenewly-selected camera.

The embodiments described herein mainly refer to video surveillance ofindividuals based on location measurements performed on the mobileterminals they carry. Alternatively, however, the methods and systemsdescribed herein can also be used to perform video surveillance (and inparticular camera hand-off) on various other types of objects. Trackingsuch objects may be performed using various other kinds of locationindications. For example, the methods and systems described herein canbe used with object fitted with Radio Frequency Identification (RFID)tags, or Automatic Vehicle Location (AVL) or Automatic Person Location(APL) transponders. In these embodiments, location measurements of anRFID tag or transponder can be used as location indications of theobject.

The embodiments described herein refer mainly to automatic selection ofcamera, i.e., camera hand-off. Additionally, when a given camera has anadjustable field-of-view (e.g., a PTZ camera), server 70 may also adjustthe selected camera's field-of-view based on the location indications,in order to best view the user.

It will thus be appreciated that the embodiments described above arecited by way of example, and that the present disclosure is not limitedto what has been particularly shown and described hereinabove. Rather,the scope of the present disclosure includes both combinations andsub-combinations of the various features described hereinabove, as wellas variations and modifications thereof which would occur to personsskilled in the art upon reading the foregoing description and which arenot disclosed in the prior art.

1. A method, comprising: receiving video images from a pan-tilt-zoom(PTZ) camera in a video surveillance system, the video images of ageographic region including an individual; probing signaling informationassociated with a wireless communication terminal possessed by theindividual; extracting the individual's location in the geographic areafrom the monitored signaling information; and adjusting, automatically,the field-of-view of the PTZ camera based on the extracted individual'slocation to better view the individual.
 2. The method according to claim1, wherein the probing signaling information comprises: passivelymonitoring information transmitted on a network of which the wirelesscommunication terminal is part.
 3. The method according to claim 2,wherein the wireless communication terminal is a cellular phone and thenetwork is a cellular network.
 4. The method according to claim 3,wherein the cellular phone is on but not in call operation.
 5. Themethod according to claim 1, further comprising: repeating the probing,extracting, and adjusting to track the individual moving through thegeographic region.
 6. The method according to claim 5, furthercomprising: switching to a second PTZ camera in the video surveillancesystem based on the extracted individual's location; probing signalinginformation associated with the wireless communication terminalpossessed by the individual; extracting the individual's location in thegeographic area from the monitored signaling information; and adjusting,automatically, the field-of-view of the second PTZ camera based on theextracted individual's location to track the individual moving through asecond geographic region.
 7. A video surveillance system, comprising: alocation tracking subsystem receiving signaling information associatedwith a wireless communication terminal possessed by an individual andextracting the individual's location in a geographic area from themonitored signaling information; a PTZ camera having an adjustablefield-of-view capable of capturing video of the geographic location; anda network video server communicatively coupled to the location trackingsubsystem and the PTZ camera, wherein the network video server adjuststhe PTZ camera's field-of-view based on the extracted individual'slocation to better view the individual.
 8. The video surveillance systemaccording to claim 7, wherein the signaling information comprisesinformation transmitted on a network of which the wireless communicationterminal is part.
 9. The video surveillance system according to claim 8,wherein the wireless communication terminal is a cellular phone and thenetwork is a cellular network.
 10. The video surveillance systemaccording to claim 9, wherein the cellular phone is on but not in calloperation.
 11. The video surveillance system according to claim 7,wherein network video server repeatedly adjusts the PTZ camera'sfield-of-view based on the extracted individual's location to track theindividual through the geographic area.
 12. The video surveillancesystem according to claim 11, the video server adjusts the field of viewof a second PTZ camera based on the extracted individual's location totrack the individual through a second geographic area.
 13. Anon-transitory tangible computer readable medium containing computerreadable instructions that when executed by a processor of a computercause the computer to perform a method comprising: receiving videoimages from a pan-tilt-zoom (PTZ) camera in a video surveillance system,the video images including an individual moving through a geographicregion; probing signaling information associated with a wirelesscommunication terminal possessed by the individual; extracting theindividual's location in the geographic area from the monitoredsignaling information; and adjusting, automatically, the field-of-viewof the PTZ camera based on the extracted individual's location to betterview the individual.
 14. The non-transitory tangible computer readablemedium according to claim 13, wherein the probing signaling informationcomprises: passively monitoring information transmitted on a network ofwhich the wireless communication terminal is part.
 15. Thenon-transitory tangible computer readable medium according to claim 14,wherein the wireless communication terminal is a cellular phone and thenetwork is a cellular network.
 16. The non-transitory tangible computerreadable medium according to claim 15, wherein the cellular phone is onbut not in call operation.
 17. The non-transitory tangible computerreadable medium according to claim 13, further comprising: repeating theprobing, extracting, and adjusting to track the individual movingthrough the geographic region.
 18. The non-transitory tangible computerreadable medium according to claim 17, further comprising: switching toa second PTZ camera in the video surveillance system based on theextracted individual's location; probing signaling informationassociated with the wireless communication terminal possessed by theindividual; extracting the individual's location in the geographic areafrom the monitored signaling information; and adjusting, automatically,the field-of-view of the second PTZ camera based on the extractedindividual's location to track the individual moving through a secondgeographic region.